High-availability Cluster
   HOME

TheInfoList



OR:

High-availability clusters (also known as HA clusters, fail-over clusters) are groups of
computer A computer is a machine that can be programmed to Execution (computing), carry out sequences of arithmetic or logical operations (computation) automatically. Modern digital electronic computers can perform generic sets of operations known as C ...
s that support
server Server may refer to: Computing *Server (computing), a computer program or a device that provides functionality for other programs or devices, called clients Role * Waiting staff, those who work at a restaurant or a bar attending customers and su ...
applications Application may refer to: Mathematics and computing * Application software, computer software designed to help the user to perform specific tasks ** Application layer, an abstraction layer that specifies protocols and interface methods used in a c ...
that can be reliably utilized with a minimum amount of down-time. They operate by using
high availability software High availability software is software used to ensure that systems are running and available most of the time. High availability is a high percentage of time that the system is functioning. It can be formally defined as (1 – (down time/ total ti ...
to harness redundant computers in groups or clusters that provide continued service when system components fail. Without clustering, if a server running a particular application crashes, the application will be unavailable until the crashed server is fixed. HA clustering remedies this situation by detecting hardware/software faults, and immediately restarting the application on another system without requiring administrative intervention, a process known as
failover Failover is switching to a redundant or standby computer server, system, hardware component or network upon the failure or abnormal termination of the previously active application, server, system, hardware component, or network in a computer net ...
. As part of this process, clustering software may configure the node before starting the application on it. For example, appropriate file systems may need to be imported and mounted, network hardware may have to be configured, and some supporting applications may need to be running as well. HA clusters are often used for critical
databases In computing, a database is an organized collection of data stored and accessed electronically. Small databases can be stored on a file system, while large databases are hosted on computer clusters or cloud storage. The design of databases spa ...
, file sharing on a network, business applications, and customer services such as
electronic commerce E-commerce (electronic commerce) is the activity of electronically buying or selling of products on online services or over the Internet. E-commerce draws on technologies such as mobile commerce, electronic funds transfer, supply chain manageme ...
websites A website (also written as a web site) is a collection of web pages and related content that is identified by a common domain name and published on at least one web server. Examples of notable websites are Google, Facebook, Amazon, and Wikipe ...
. HA cluster implementations attempt to build redundancy into a cluster to eliminate single points of failure, including multiple network connections and data storage which is redundantly connected via
storage area network A storage area network (SAN) or storage network is a computer network which provides access to consolidated, block-level data storage. SANs are primarily used to access data storage devices, such as disk arrays and tape libraries from serve ...
s. HA clusters usually use a heartbeat private network connection which is used to monitor the health and status of each node in the cluster. One subtle but serious condition all clustering software must be able to handle is
split-brain Split-brain or callosal syndrome is a type of disconnection syndrome when the corpus callosum connecting the two hemispheres of the brain is severed to some degree. It is an association of symptoms produced by disruption of, or interference wit ...
, which occurs when all of the private links go down simultaneously, but the cluster nodes are still running. If that happens, each node in the cluster may mistakenly decide that every other node has gone down and attempt to start services that other nodes are still running. Having duplicate instances of services may cause data corruption on the shared storage. HA clusters often also use
quorum A quorum is the minimum number of members of a deliberative assembly (a body that uses parliamentary procedure, such as a legislature) necessary to conduct the business of that group. According to ''Robert's Rules of Order Newly Revised'', the ...
witness storage (local or cloud) to avoid this scenario. A witness device cannot be shared between two halves of a split cluster, so in the event that all cluster members cannot communicate with each other (e.g., failed heartbeat), if a member cannot access the witness, it cannot become active.


Application design requirements

Not every application can run in a high-availability cluster environment, and the necessary design decisions need to be made early in the software design phase. In order to run in a high-availability cluster environment, an application must satisfy at least the following technical requirements, the last two of which are critical to its reliable function in a cluster, and are the most difficult to satisfy fully: * There must be a relatively easy way to start, stop, force-stop, and check the status of the application. In practical terms, this means the application must have a command line interface or scripts to control the application, including support for multiple instances of the application. * The application must be able to use shared storage (
NAS Nas (born 1973) is the stage name of American rapper Nasir Jones. Nas, NaS, or NAS may also refer to: Aviation * Nasair, a low-cost airline carrier and subsidiary based in Eritrea * National Air Services, an airline in Saudi Arabia ** Nas Air ( ...
/ SAN). * Most importantly, the application must store as much of its state on non-volatile shared storage as possible. Equally important is the ability to restart on another node at the last state before failure using the saved state from the shared storage. * The application must not corrupt data if it crashes, or restarts from the saved state. * A number of these constraints can be minimized through the use of virtual server environments, wherein the hypervisor itself is cluster-aware and provides seamless migration of virtual machines (including running memory state) between physical hosts—se
Microsoft Server 2012 and 2016 Failover Clusters
** A key difference between this approach and running cluster-aware applications is that the latter can deal with server application crashes and support live "rolling" software upgrades while maintaining client access to the service (e.g. database), by having one instance provide service while another is being upgraded or repaired. This requires the cluster instances to communicate, flush caches and coordinate file access during hand-off. Failover clustering falls under planning, creating and configuring also troubleshooting.


Node configurations

The most common size for an HA cluster is a two-node cluster, since that is the minimum required to provide redundancy, but many clusters consist of many more, sometimes dozens of nodes. The attached diagram is a good overview of a classic HA cluster, with the caveat that it does not make any mention of quorum/witness functionality (see above). Such configurations can sometimes be categorized into one of the following models: * Active/active — Traffic intended for the failed node is either passed onto an existing node or load balanced across the remaining nodes. This is usually only possible when the nodes use a homogeneous software configuration. * Active/passive — Provides a fully redundant instance of each node, which is only brought online when its associated primary node fails. This configuration typically requires the most extra hardware. * N+1 — Provides a single extra node that is brought online to take over the role of the node that has failed. In the case of heterogeneous software configuration on each primary node, the extra node must be universally capable of assuming any of the roles of the primary nodes it is responsible for. This normally refers to clusters that have multiple services running simultaneously; in the single service case, this degenerates to active/passive. * N+M — In cases where a single cluster is managing many services, having only one dedicated failover node might not offer sufficient redundancy. In such cases, more than one (M) standby servers are included and available. The number of standby servers is a tradeoff between cost and reliability requirements. * N-to-1 — Allows the failover standby node to become the active one temporarily, until the original node can be restored or brought back online, at which point the services or instances must be failed-back to it in order to restore high availability. * N-to-N — A combination of active/active and N+M clusters, N to N clusters redistribute the services, instances or connections from the failed node among the remaining active nodes, thus eliminating (as with active/active) the need for a 'standby' node, but introducing a need for extra capacity on all active nodes. The terms ''logical host'' or ''cluster logical host'' is used to describe the
network address A network address is an identifier for a node or host on a telecommunications network. Network addresses are designed to be unique identifiers across the network, although some networks allow for local, private addresses, or locally administere ...
that is used to access services provided by the cluster. This logical host identity is not tied to a single cluster node. It is actually a network address/hostname that is linked with the service(s) provided by the cluster. If a cluster node with a running database goes down, the database will be restarted on another cluster node.


Node reliability

HA clusters usually use all available techniques to make the individual systems and shared infrastructure as reliable as possible. These include: *
Disk mirroring In data storage, disk mirroring is the replication of logical disk volumes onto separate physical hard disks in real time to ensure continuous availability. It is most commonly used in RAID 1. A mirrored volume is a complete logical representat ...
(or Redundant Arrays of Independent Disks—RAID) so that failure of internal disks does not result in system crashes. The
Distributed Replicated Block Device DRBD is a distributed replicated storage system for the Linux platform. It is implemented as a kernel driver, several userspace management applications, and some shell scripts. DRBD is traditionally used in high availability (HA) computer clu ...
is one example. * Redundant
network Network, networking and networked may refer to: Science and technology * Network theory, the study of graphs as a representation of relations between discrete objects * Network science, an academic field that studies complex networks Mathematics ...
connections so that single cable, switch, or network interface failures do not result in network outages. * Redundant
storage area network A storage area network (SAN) or storage network is a computer network which provides access to consolidated, block-level data storage. SANs are primarily used to access data storage devices, such as disk arrays and tape libraries from serve ...
(SAN) connections so that single cable, switch, or interface failures do not lead to loss of connectivity to the storage (this would violate
shared nothing architecture Shared may refer to: * Sharing * Shared ancestry or Common descent * Shared care * Shared-cost service * Shared decision-making in medicine * Shared delusion, various meanings * Shared government * Shared intelligence or collective intelligence ...
). * Redundant
electrical power Electric power is the rate at which electrical energy is transferred by an electric circuit. The SI unit of power is the watt, one joule per second. Standard prefixes apply to watts as with other SI units: thousands, millions and billions of ...
inputs on different circuits, usually both or all protected by
uninterruptible power supply An uninterruptible power supply or uninterruptible power source (UPS) is an electrical apparatus that provides emergency power to a load when the input power source or mains power fails. A UPS differs from an auxiliary or emergency power system ...
units, and redundant
power supply A power supply is an electrical device that supplies electric power to an electrical load. The main purpose of a power supply is to convert electric current from a source to the correct voltage, current, and frequency to power the load. As a r ...
units, so that single power feed, cable, UPS, or power supply failures do not lead to loss of power to the system. These features help minimize the chances that the clustering failover between systems will be required. In such a failover, the service provided is unavailable for at least a little while, so measures to avoid failover are preferred.


Failover strategies

Systems that handle failures in distributed computing have different strategies to cure a failure. For instance, the
Apache Cassandra Cassandra is a free and open-source, distributed, wide-column store, NoSQL database management system designed to handle large amounts of data across many commodity servers, providing high availability with no single point of failure. Cassandr ...
API An application programming interface (API) is a way for two or more computer programs to communicate with each other. It is a type of software interface, offering a service to other pieces of software. A document or standard that describes how ...
Hector In Greek mythology, Hector (; grc, Ἕκτωρ, Hektōr, label=none, ) is a character in Homer's Iliad. He was a Trojan prince and the greatest warrior for Troy during the Trojan War. Hector led the Trojans and their allies in the defense o ...
defines three ways to configure a failover: * Fail Fast, scripted as "FAIL_FAST", means that the attempt to cure the failure fails if the first node cannot be reached. * On Fail, Try One - Next Available, scripted as "ON_FAIL_TRY_ONE_NEXT_AVAILABLE", means that the system tries one host, the most accessible or available, before giving up. * On Fail, Try All, scripted as "ON_FAIL_TRY_ALL_AVAILABLE", means that the system tries all existing, available nodes before giving up.


See also

*
Service Availability Forum The Service Availability Forum (SAF or SA Forum) is a consortium that develops, publishes, educates on and promotes open specifications for carrier-grade and mission-critical systems. Formed in 2001, it promotes development and deployment of com ...
*
OpenSAF OpenSAF (commonly styled SAF, the Service Availability Framework) is an open-source service-orchestration system for automating computer application deployment, scaling, and management. OpenSAF is consistent with, and expands upon, Service Ava ...
*
Urgent computing Urgent computing is prioritized and immediate access on supercomputers and grids for emergency computations such as severe weather prediction during matters of immediate concern. Applications that provide decision makers with information during c ...
*
Pacemaker (software) Pacemaker is an open-source high availability resource manager software used on computer clusters since 2004. Until about 2007, it was part of the Linux-HA project, then was split out to be its own project. It implements several APIs for con ...
*
IBM Parallel Sysplex In computing, a Parallel Sysplex is a cluster of IBM mainframes acting together as a single system image with z/OS. Used for disaster recovery, Parallel Sysplex combines data sharing and parallel computing to allow a cluster of up to 32 systems ...
*
List of cluster management software List of software for cluster management. Free and open source * HA ** Apache Mesos, from the Apache Software Foundation ** Kubernetes, founded by Google Inc, from the Cloud Native Computing Foundation ** Heartbeat, from Linux-HA ** Docker Swarm ...


References


Further reading

* Greg Pfister: ''In Search of Clusters'', Prentice Hall, * Evan Marcus, Hal Stern: ''Blueprints for High Availability: Designing Resilient Distributed Systems'', John Wiley & Sons, * Chee-Wei Ang, Chen-Khong Tham: ''Analysis and optimization of service availability in a HA cluster with load-dependent machine availability'', IEEE Transactions on Parallel and Distributed Systems, Volume 18, Issue 9 (September 2007), Pages 1307-1319,

{{DEFAULTSORT:High-Availability Cluster Quality control Fault-tolerant computer systems